User interface for a medical informatics system

ABSTRACT

A user interface for a medical informatics system, which permits a physician to work with digitized medical images in a manner that the physician is accustomed to working with traditional analog film, is disclosed. The user interface includes a through a patient browser view to provide the ability to select studies, which consist of medical images and series, for patients. After selecting the studies, the user, through a patient canvas view, may then organize the studies as well as the images/series within the studies, including resizing the studies and the images/series within a study. The user may also pan and zoom images to view portions of an image at various resolutions. Furthermore, the user of the user interface may analyze the image by selecting to view the image in detail in a large floating window.

BACKGROUND OF THE INVENTION

[0001] 1. Field of the Invention

[0002] The present invention is directed toward the field of medicalinformatics, and more particularly toward a user interface for a medicalinformatics system.

[0003] 2. Art Background

[0004] Radiology equipment (e.g., CT scanners, MRI scanners, X-Ray etc.)is in wide spread use as diagnostic tools in hospitals today.Traditionally, radiology departments utilize equipment, such as X-Raymachines, that generate images on film. Typically, when collectinginformation from a diagnostic tool, several medical images are generatedfor subsequent analysis and diagnosis of the patient's medicalcondition. This collection of medical images may be referred to as a“study.” For example, a study from an X-Ray machine may consist of anumber of X-Rays taken from different perspectives of the target area.It is the totality of the study that the physician uses to make adiagnosis of the patient.

[0005] It has become more common in the medical field for images to bestored, distributed, and viewed in digital form using computertechnology. Currently, Picture Archival and Communication Systems orPACS have been in widespread use. In a typical PACS application, imagedata obtained by imaging equipment such as CT scanners or MRI scannersare stored in the form of computer data files. The size of a data filefor an image varies depending on the size and resolution of the image.For example, a typical image file for a diagnostic-quality chest X-rayis on the order of 10 megabytes (MB). The image data files are usuallyformatted in a “standard” or widely accepted format. In the medicalfield, one widely used image format is known as DICOM. The DICOM imagedata files are distributed over computer networks to specialized viewingstations capable of converting the image data to high-resolution imageson a CRT display.

[0006] The digitized medical images potentially provide to the medicalcommunity advancements due to the ability to electronically store,transfer and view digitized images over geographically disparate areas.However, prior art systems for viewing the digital data do not comportwith how physicians traditionally operate. Physicians have becomeaccustomed to working with analog film. First, to conduct a diagnosesusing traditional film, the physician chooses the films for a patientthat will aid in the analysis of the patient's condition. From theselected films, the physician organizes the films in a manner suitableto conduct the analysis and subsequent diagnoses. Specifically, the filmis placed on a light board for viewing. The light board projects lightthrough the film so that the physician may read the image imposed on thefilm. Prior to analyzing a study, a physician may organize the physicalsheets of film on the light board in a manner suitable for conductingthe analysis. It may be advantageous for a physician to place, on thelight board, two sheets of film next to one another in order to analyzea condition relative to the two films. For example, the first film maycomprise data taken at an earlier date, whereas the second film maycontain data recently obtained. By placing the films side-by-side, thephysician may analyze how a particular condition has changed over time.

[0007] Prior art systems for viewing digitized medical images do notprovide a means to operate in a manner in which physicians work. Asillustrated by the above example, using these prior art systems, aphysician is not permitted to effectively organize medical images in amanner in which physicians may organize traditional analog films.Accordingly, it is desirable to develop a user interface for a medicalinformatics system that emulates the way a physician works by providingmaximum flexibility for the physician to select, organize, navigate andsubsequently analyze medical images. Furthermore, prior art systems forviewing digitized medical images display static images, in that the useris not permitted to navigate (i.e., pan or zoom the original image).Accordingly, it is also desirable to generate a system that permits“dynamic” interaction with medical images to provide the physician withmaximum flexibility to interact with the image.

SUMMARY OF THE INVENTION

[0008] A user interface for a medical informatics system permits aphysician to work with digitized medical images in a manner that thephysician is accustomed to working with traditional analog film. Theuser interface provides the user to ability to select studies, whichconsist of medical images and series, for patients. In one embodiment,the user selects studies on the user interface through a patient browserview. After selecting the studies, the user may then organize thestudies as well as the images/series within the studies, includingresizing the studies and the images/series within a study. The user mayalso navigate around the images/series. Specifically, the user has theability to pan and zoom images to view portions of an image at variousresolutions. Furthermore, the user of the user interface may analyze theimage by selecting to view the image in detail in a large floatingwindow. In one embodiment, the organization, navigation, and analysis ofstudies and images/series are performed through a patient canvas view.

[0009] In one embodiment, the patient canvas view is displayed in astandard orientation such that each horizontal scroll bar contains astudy. For this embodiment, each study displayed on the patient canvasview is broken out left to right into one or more series for CT/MR andone or more images for CD/DR. The user, using a horizontal scroll bar ispermitted to scroll left and right to display the series/imagescontained within the study. Multiple studies are laid out from top tobottom on the patient canvas view. A single vertical scroll bar isprovided to permit the user to scroll, in a vertical direction (i.e.,from top to bottom), to display the multiple studies. Using the userinterface, the user may organize studies by re-arranging the relativevertical positions among the studies. Thus, the studies (i.e., thewindow encompassing the studies) may be re-sized to any user-desiredsize. The user may also use the features of the patient canvas view toorganize images, within a study, by re-arranging the relative horizontalpositions among the images/series within a study via a drag and dropoperation.

BRIEF DESCRIPTION OF THE DRAWINGS

[0010]FIG. 1 illustrates one embodiment for an initial patient browserview.

[0011]FIG. 2 illustrates an example patient browser view with aplurality of the studies for each patient.

[0012]FIG. 3 illustrates an example display of a patient browser viewthat includes selection of patient studies.

[0013]FIG. 4 illustrates an example patient canvas view in accordancewith one embodiment of the present invention.

[0014]FIG. 5 illustrates a user operation to scroll images within astudy.

[0015]FIG. 6 illustrates the patient canvas view subsequent to a useroperation that scrolls among studies.

[0016]FIG. 7 illustrates one embodiment of a medical informatics systemfor use with the user interface of the present invention.

[0017]FIG. 8a illustrates an example of a pyramidal data structure.

[0018]FIG. 8b illustrates level three and level four decompositions forthe 4K×4K source image of FIG. 8a.

DETAILED DESCRIPTION

[0019] The user interface of the medical informatics system provides aubiquitous viewing environment for fast and simple access to medicalimages across the enterprise. The user interface may be operated by aphysician in manner in which physicians are accustomed to working withtraditional analog film. First, the user of the medical informaticssystem may select studies, which consist of medical images/series, forpatients. In one embodiment, this functionality is provided through apatient browser view. The user may then organize the studies as well asthe images/series within the studies, including resizing the studies andthe images/series within a study. The user may also navigate around theimages/series. Specifically, the user has the ability to pan and zoomimages to view portions of an image at various resolutions. Furthermore,the user of the user interface may analyze the image by selecting toview the image in detail in a large floating window. In one embodiment,the organization, navigation, and analysis of studies and images/seriesare performed through a patient canvas view. Accordingly, the userinterface of the present invention emulates the way a physician workswith medical images by providing full capabilities to select, organize,navigate and analyze medical information.

[0020] In one embodiment, the user interface consists of primarily asingle window interface. However, additional floating windows aregenerated, when appropriate, to provide detailed image viewing. For thesingle window interface embodiment, tabs are presented to the user topermit the user to navigate between a patient browser view and one ormore patient canvas views. In general, the patient browser view permitsthe user to select studies for one or more patients. A study, specificto a patient, comprises images obtained from a diagnostic tool, and insome cases, additional information (e.g., medical report) to augment theimage data. The studies define the repository of medical images that maybe used during the session. The patient canvas view provides a screensurface area for the organization, navigation and analysis of thepatient medical information selected.

[0021] In one embodiment, the user interface presents the user with asimple login window. Using this login window, the user may enter a username and password. If the user is successfully authenticated by theserver (e.g., image server 720, FIG. 7), then the main window of theclient computer is displayed with the patient browser tab selected.

[0022] In one embodiment, the user interface operates as a plug-in withan Internet browser application, such as Microsoft Internet Explorer orNetscape Navigator. In one embodiment, the user interface comprises, inpart, executable software configured as a Microsoft® ActiveX Control.For this embodiment, the ActiveX Control is a “plug-in” to a Web browserapplication. The Internet browser application includes a title bar(e.g., title bar 102, FIG. 1) including controls to minimize, maximizeand close the browser application, as well as a tool bar (e.g., tool bar104, FIG. 1). Although the user interface is shown herein as an Internetbrowser plug-in, the user interface may operate independent of otherapplication programs without deviating from the spirit or scope of theinvention.

[0023] Patient Browser View for Selecting Studies:

[0024]FIG. 1 illustrates one embodiment for an initial patient browserview. For this embodiment, following user login, the user interfaceopens and displays a patient browser tab, labeled 106 on FIG. 1. In oneembodiment, the user interface 100 contains search capabilities topermit a user to locate and select medical information for one or morepatients. For this feature, the user interface 100 contains, as part ofthe patient browser tab view, controls and entry boxes (122) to allowsearching for patients and studies. Specifically, entry boxes forsearching include: patient's name 110, patient ID 116, patient location118, and date of last exam 120. The user interface 100 also permitssubmission of predefined queries for the fields: physician, patientlocation, physician group, and body part. These predefined queries arestored as part of a user profile. The physician group is a class thatgroups different types of physicians (e.g., neurology, orthopedic,oncology, etc.). The physician groups may be assigned by anadministrator of the medical informatics system. If the predefined queryoccurs as part of the user login process, then the initial state of thepatient browser displays the results of that query. Alternatively, if alogin query is not found or available, then there is no content in thepatient list display area.

[0025] In general, the patient browser list view 100 displays a list ofpatients and their corresponding studies. Information on patients andtheir studies is displayed in the area labeled 128 in the user interfacewindow 100. The example display of the FIG. 1 displays, in a patientlist display area, information for two patients, Jamie Walter 124 andCharles Wilkins 126. A patient ID, corresponding to the patient's name,is also displayed. For this embodiment, the list of studies onlyindicates the specific study, and does not indicate the series or imagecontained in that study. As shown in FIG. 1, the information displayedfor each patient includes: last name 110, first name 112, middle initial114, patient ID 116, patient location 118, and date of last exam 120(derived from the most recent study). In addition, the user may sort thelist of patients by last name, patient location, and date of last exam.

[0026]FIG. 2 illustrates an example patient browser view with aplurality of the studies for each patient. To obtain the patient browserview of FIG. 2 from the patient browser view of FIG. 1, the user, usinga cursor control device, “clicks” on a patient name line (e.g., patientname line 124 for Jamie Walter and patient name line 126 for CharlesWilkins), and the studies available for that patient are displayed. Asshown in FIG. 2, a tree paradigm is used to display the studies beneaththe patient title bars 124 and 126. For the hierarchical patient studydisplay of the FIG. 2, the display line for each study includes: a checkbox to indicate selection status, modality, study description, accessionnumber, and exam date. Specifically, the example of FIG. 2 shows, forthe patient “Walter Jamie”, the studies labeled 130, 132 and 134 on FIG.2. The abbreviation “MR” connotes magnetic resonance, and theabbreviation “CR” connotes conventional radiography (e.g., an X-Ray).For the patient “Charles Wilkins”, a plurality of CT studies arerevealed.

[0027] If the user clicks, using a cursor control device, to select astudy, then the selection check mark in the check boxes 136 and 138 aretoggled. FIG. 3 illustrates an example display of a patient browser viewthat includes selection of patient studies. For this example, the userselected, for the patient “Charles Wilkins”, CT studies 140, 142, 146,157, 159 and 160. The selection of the studies are indicated by thecheck mark in the check box adjacent to the study description (e.g.,CT). This selection response adds and or subtracts studies from thecurrent selection for subsequent display in the patient canvas view. Inother embodiments, additional user interface features for the patientbrowser view permit ease of selecting and deselecting studies. Forexample, the key strokes “shift—click” executed by the user selects acontiguous range of studies. The key strokes “control—click” deselectsall other studies and selects the single study.

[0028] As study selections are made by the user from the patient browserview, the user interface creates tabs for each patient that has at leastone study selected. For the example shown in FIG. 3, patient tabs 150and 155 are displayed for the patients “Walter Jamie” and “CharlesWilkins”, respectively. The tabs are created on a per patient basis, onetab for each patient with selected studies. For this embodiment, thetabs are displayed from left to right in an order dictated by thecurrent sort order. The example of FIG. 3 shows sorting of the patient'slast name in alphabetical order. The user may move to the patient canvasview (described below) for that patient by selecting the correspondingtab. As a shortcut, if the user “double clicks” on a patient's name(e.g., line 126 for Charles Wilkins and line 124 for Jamie Walter), acanvas tab is created for the patient, and that tab is displayed similarto tabs 150 and 155 on FIG. 3. As a shortcut to adding the canvas tabfor the patient and selecting all studies for display on the patientcanvas view, the user “double-clicks” on a study in the list. If allstudies are selected during a double-click, then each of the selectedstudies are displayed within the canvas view.

[0029] Patient Canvas View for Organization of Studies and Images:

[0030] In general, the patient canvas view of the user interface permitsa user to organize, navigate, and analyze images/series in the studiesselected. FIG. 4 illustrates an example patient canvas view inaccordance with one embodiment of the present invention. A patientcanvas view 200 includes a plurality of studies for the selectedpatient, “Jamie, Walter.” As shown in FIG. 4, the patient tab, labeled150 for “Jamie, Walter” is highlighted. Each tab displayed has acorresponding patient canvas view. Thus, another patient canvas viewexists for the patient “Charles Wilkins.”

[0031] The area beneath the displayed tabs is the primary display areafor the studies and series/images. For the example of FIG. 4, twostudies, arranged vertically on the screen, are shown. In oneembodiment, selected studies are automatically laid out from top tobottom on the patient canvas view. Each study is broken out left toright into one or more series for CT/MR and one or more images forCD/DR. In the example of FIG. 4, the first or top study includes theseries of images labeled 230, 235 and 240 on FIG. 4. The second study,displayed on the bottom of the patient canvas view, currently displaysthe three images: 260, 265, and 270.

[0032] In one embodiment, the patient canvas view is displayed in astandard orientation such that each horizontal scroll bar (scroll bar110 for the top study) contains a study. The user, using the horizontalscroll bar (e.g., horizontal scroll bar 110), is permitted to scrollleft and right to display the series/images contained within the study.Also, a single vertical scroll bar (e.g., vertical scroll bar 205 onFIG. 4) is provided to permit the user to scroll, in a verticaldirection (i.e., from top to bottom), to display multiple studies.Furthermore, the height of each study may be varied within the patientcanvas view. To accomplish this operation, the user, using a cursorcontrol device, places the cursor on a horizontal grab bar on the study(e.g., bar 290 for the top study and bar 295 for the bottom study), andresizes the study to the appropriate height. Using this technique, thestudies (i.e., the window encompassing the studies), may be resized.

[0033] Using the user interface, the user may organize studies byre-arranging the relative vertical positions among the studies. The usermay also use the features of the patient canvas view to organize images,within a study, by rearranging the relative horizontal positions amongthe images/series within a study. In one embodiment, these organizationoperations are executed via a drag and drop operation. As is well known,in a drag and drop operation, the user “selects” a series/image orstudy, and drags the series/image or study to the destination location.When the image is located at the destination location, the user releasesthe series/image or study to complete the drag and drop operation. Acontrol “hot area” at the left side of each study row is displayed toprovide a handle for the user to grab the study in the drag and dropoperation. The study “handle” is labeled 275 for the top study and islabeled 280 for the bottom study of the FIG. 4. The series (CT/MR) andimages (CR/DR) may also be re-arranged within a study (i.e., re-arrangerelative horizontal positions) using the drag and drop operation. Forthis operation, the user may “grab” an image or series using the titlebar or annotation area, such as title bar 220 for series 235 on FIG. 4.The drag and drop operation provides maximum flexibility for the user toarrange the patient canvas view in any manner desired by the user.

[0034] The position of studies and images displayed on the patientcanvas view may also be arranged by user execution of a cut and pasteoperation. As is well-known for a general cut and paste operation, theuser selects the study (e.g., using the cursor control device orentering a keystroke sequence), executes the cut operation with theappropriate keystroke, re-positions the cursor with the cursor controldevice in the new destination location, and executes the “paste”command.

[0035] In one embodiment, a rule set is applied to analyze a study ofseries/images displayed in a single row to determine the proper heightfor the row. Using this technique, row heights are selected based on thenearest optimal representation. The default target row height is 320pixels. The actual row height is determined by analyzing the rowcontents (i.e., series/images) so that unnecessary space is eliminated.In another embodiment, the row height is saved as a user preference, anda target row height is used to display studies for that user. In otherembodiments, the target row height is determined from the user's screensize or window resolution.

[0036] The patient browser view on the user interface provides thefunctionality to “clone” an image. To this end, a user may copy an imageor series, and paste the image or series in a different location. Forthe example shown in FIG. 4, the user may copy, through a standard copyoperation, image 260 in the second study (i.e., the bottom study), andpaste the image to the right of image 265. In other embodiments, theuser may copy an image in one study (e.g., the bottom study), and pastethe image into a different study (e.g., the first or top study). Theresult of this operation is shown on the display of FIG. 5, startingwith the display of FIG. 4, with image 260 appearing in both the firstand second studies.

[0037]FIG. 5 illustrates a user operation to scroll images within astudy. To view additional images contained in the first study shown inFIG. 4, the user, utilizing scroll bar 210, scrolls through theimages/series contained within the study. FIG. 5 shows a different viewfrom the study of FIG. 4 subsequent to a user operation to scroll theimages/series from right to left.

[0038]FIG. 6 illustrates a user operation to scroll among studies. FIG.6 illustrates the patient canvas view subsequent to a user operationthat scrolls among studies. Specifically, for this example, the user,utilizing the scroll bar 205, scrolls, in a vertical direction (e.g.,from bottom to top), the top and bottom studies to view more of thebottom study (and subsequently less of the top study).

[0039] In one embodiment, each series/image displayed within a studyincludes control points and annotation information. The control pointsand annotation information may be implemented similar to a standardwindow in a user interface. For example, the study “date and time” maybe displayed at the top of the image, and the study and seriesdescriptive information may be displayed below the image. For theexample of FIG. 4, the image 230 of the top study includes, as a controlpoint, the bar with the text “test 3”, labeled 215, and an annotationfield 245 entitled “annotation test 3.” The annotation field may includeany type of information used to describe the image, including imageframe and canvas row frame information. In addition, for floatingwindows (described below), annotation information such as patient nameand date of study may be displayed.

[0040] After a session, the selection and arrangement of studies andimages/series are stored in a persistent datastore. When that userselects the same patient again, the patient browser view is restored tothe previous display from the prior session.

[0041] Patient Canvas View for Navigation and Analysis of Images:

[0042] The patient canvas view of the user interface permits a user tofully “navigate” the image. Typically, medical images are large, andcannot be displayed at full resolution on a computer monitor. Thus, whendisplayed in small windows (e.g., image 235 in FIG. 4 displayed atapproximately 320 pixels), only portions of the medical image aredisplayed at any one time. For example, a medical image consisting of apixel resolution of 4K×4K cannot be displayed at full resolution on amonitor comprising a pixel resolution of 1024×768. Thus, only portionsof the 4096×4096 source image are displayed through the user interfaceat a given time. For example, the user interface may display, in a512×512 window, the entire source image at a lower resolution (i.e., athumbnail sketch of the image).

[0043] The images, displayed on the user interface, are “dynamicimages.” The images are dynamic because the user may fully manipulateeach image to display different portions of the image (pan the originalimage) at different resolutions (zoom in and out). In one embodiment, adynamic transfer syntax, described below, provides full functionality toallow the user to manipulate the image in any manner desired. Startingwith the lower resolution “dynamic image”, the user may zoom-in on amore specific portion of the image. Thereafter, the user may pan theimage to view a different portion of the image at the higher resolution.Accordingly, through the pan and zoom functions, the user may navigatethrough the images.

[0044] In one embodiment, only portions of an image are displayed as theuser continuously pans an image. The eye is only capable of perceiving acertain level of detail while pixels are moving during the panoperation. The user interface takes advantage of this fact and only usesa lower resolution version of the image during the pan operation. Thelower resolution version provides adequate detail for user perception.When the continuous panning activity halts, additional details aresupplied to the image to display the image at the desired resolution.This feature enhances performance of the medical informatics system inlow network bandwidth implementations.

[0045] The patient canvas view on the user interface permits a user tolink series within the canvas. With this feature, as a user scrollsthrough slices of a first series, the second series, linked to the firstseries, is also scrolled. The patient canvas view also permits linkingof any image or series, including images and series displayed infloating windows. The user interface also permits a user to clone aseries for display at different window widths and window levels(“WW/WL”) (i.e., contrast and brightness, respectively). The userinterface further permits a user to scroll a CT/MR series to aparticular slice, and then link this series to another series forsimultaneous cine. In addition, the patient canvas view maintains, forsimultaneous cine between two series, the same anatomical position forboth series, even if the series contains a different number of slices.For example, a first series may contain 100 slices within an anatomicalposition of a patient, and a second series may contain only 10 sliceswithin the same anatomical position of the patient. For this example,the simultaneous cine feature displays 10 slices of the first series forevery 1 slice of the second series. WW/WL acted upon any of the threedisplay modes is inherited by subsequent display of that series or imageduring the current session. This includes larger windows created for aseries or image. Multiple link channels are supported, as indicated by anumber by a link icon and a drop down selection option at the point oflinking. For this embodiment, the user may move through a single or linkseries with a scroll wheel on a cursor control device, pan and zoomaround CR/DR images, and use the left button of the cursor controldevice to change WW/WL.

[0046] The patient canvas view of the user interface permits a user tofully “analyze” the image. For the analysis stage, the user interface ofthe present invention permits a user to create detailed views ofselected images. In one embodiment, the user interface for the medicalinformatics system permits the user to generate large floating windowsfor detailed views. The detailed views permit a physician to analyze theimage once the desired portion of the image is located in the navigationphase. For example, if the user navigates to a specific portion of alarge image, the user may invoke the user interface to display thedetailed portion of the image in a large floating window size to capturethe full resolution of the specific portion. In one embodiment, tocreate a floating window, the user double-clicks on the image, using thecursor control device, and the image is displayed in the large floatingwindow. In one embodiment, the full floating window consists ofapproximately 75 percent of the display area. For example, the floatingwindow target height may be 640 pixels, as compared to the study scrollarea target height of 320 pixels. For standard CT/MR, the display areamay comprise a 512×512 pixel window.

[0047] Linked images and series may also be displayed in floatingwindows. In one embodiment for implementing this feature, the userdouble clicks on a linked image, or selects multiple series/images, toreceive a display of a collage of those series/images, each displayed asa floating window. In this view, the user may cine through the series,as described above, linked or unlink two series, change the WW/WL withthe right mouse button, or pan and zoom with a single or linked CT/DRimages.

[0048] A double-click at the floating window level or zoom box controltakes the user directly to the full screen display mode with the sameimage manipulation interactions. On a floating unlinked CT/MR serieswindow, a double click cursor action by the user brings the images up toa nine-on-one tile mode. The nine-on-one tile mode displays images ontop of one another. Under this scenario, the user may use the scrollwheel to move the tile images back and forth one page at a time. Aleft-hand button on the cursor control device permits the user to WW/WLupon all the displayed images, so as to maintain persistence whilescrolling the pages. In one embodiment, for the nine-on-one tile mode,the size of each image within the window may comprise 256×256 pixels. Inone embodiment, floating windows have basic intelligent layoutproperties, in that multiple floating windows stack using an offset ofapproximately 16 pixels to the right and 100 pixels down. As anadditional feature, the floating windows have a button at the base ofthe window for prior image/series and next image/series control. Also,the floating windows have a link item menu available on the lower rightcorner of the image area, similar to the link button on the canvas. Thisallows shared pan zoom for plain images, shared cine for CT/MR seriesand shared page by page review for CT/MR series in tiled mode.Additionally, the user interface permits the user to toggle, using thecursor control device and keys, functionality between pan/zoom and slicenavigation for CT/MR images.

[0049] In one embodiment, the user interface displays, in addition toimage/series and studies, radiological reports to the left side of eachstudy display area. The report area is associated with the study, and ascroll bar permits the user to scroll vertically to read the textcontained in the report. In one embodiment, the report feature includesa “splitter” control bar so as to allow the user to adjust thehorizontal display area of the report area to expand or contract thesize. Example splitter control bars 275 and 280 are shown in FIG. 4.

[0050] Medical Informatics System:

[0051]FIG. 7 illustrates one embodiment of a medical informatics systemfor use with the user interface of the present invention. In oneembodiment, a medical informatics system employs dynamic transfersyntax. For this embodiment, medical informatics system 700 includesimaging equipment 705 to generate source images 715 for storage inelectronic form in an image archive 712. The image archive 712 containselectronic storage components such as disk drives and tape drives usedto store the images in a highly reliable manner. The images are storedin a suitable archival format, such as the above-mentioned DICOM format.The imaging equipment 705 includes any type of equipment to generateimages, including medical equipment (e.g., X-ray equipment, CT scanners,and MR scanners).

[0052] For this embodiment, the medical informatics system 700 includesat least one image server 720. The pyramidal data structure is stored inimage server 720. Image server 720 is coupled to one or more clientcomputers via a direct or network connection, labeled 780 on FIG. 7. Theuser interface of the present invention operates on client computers.However, the user interface may operate as a server application thatprovides functionality to the client computers. For the example shown inFIG. 7, client computers include both thick clients (i.e., a computerwith robust processing, memory, and display resources), as well as thinclients (i.e., a computer with minimal processing, memory, and displayresources). Specifically, for the example embodiment of FIG. 7, a clientcomputer 740 consists of a workstation, client computers 750 and 760consist of desktop computers, and client computers 770 consist of aportable or notebook computer.

[0053] For this embodiment, the image server 720 transmits to the clientcomputers 740, 750 and 760 transformations of the source image 715(“transform data”), stored as pyramidal data structure 735, to re-createimages and sub-images in the client computers. The image server 720transfers only the coefficient data required to reconstruct a requestedimage at the client(s), thus implementing a “just in time” data deliverysystem. The dynamic transfer syntax technique permit use of a networkwith moderate bandwidth capacity, while still providing low latency fortransfer of large data files from the image server 720 to clientcomputers 740, 750, 760 and 770. For example, the network 780 in themedical informatics system 700 may utilize an Ethernet (10baseT) mediumor an ISDN transmission medium. Regardless, any network, including widearea networks (WANs) and local area networks (LANs) may be used withoutdeviating from the spirit and scope of the invention. The medicalinformatics system 700 processes one or more source images 715.Generally, the source image(s) 715 includes a digitized medical imagegenerated from medical instrumentation (e.g., mammogram, X-Ray, MRI,CATSCAN, etc.). Although the present invention is described for use withmedical images, any large data file may be used as a source image 115without deviating from the spirit or scope of the invention.

[0054] As shown in FIG. 7, the source image(s) 715 are input todecomposition processing 125. In general, decomposition processing 125transforms the source images 715 into the dynamic transfer syntaxrepresentation, also referred to herein as pyramidal data structure 735.In general, the pyramidal data structure 735 comprises a hierarchicalrepresentation of the source image. Each level of the hierarchicalrepresentation is sufficient to reconstruct the source image at a givenresolution. In one embodiment, the decomposition processing 725 utilizesa sub-band decomposition to generate the hierarchical representation. Ingeneral, sub-band decomposition consists of executing a process toseparate “high-pass” information from “low-pass” information. For thesub-band decomposition embodiment, decomposition processing 125comprises a finite impulse response (FIR) filter.

[0055] In one embodiment that uses sub-band decomposition, thedecomposition processing 125 uses wavelet transforms, which are asub-class of the sub-band decomposition transform. In general, thewavelet transform may be selected so that the kernels aggregate asufficient amount of the image information into the terms orcoefficients. Specifically, the information is aggregated into the “lowlow” component of the decomposition. In one embodiment, kernels of thewavelet transform are selected so as to balance the computationalefficiency of the transform with optimization of the aggregateinformation in the low pass components. This characteristic of wavelettransforms permits transfer, and subsequent display, of a goodrepresentation of the source image at a particular resolution whilemaintaining the computational efficiency of the transform.

[0056] The wavelet transform function embodiment generatesmathematically independent information among the levels of thehierarchical representation. Accordingly, there is no redundantinformation in the pyramidal data structure 735. Thus, pyramidal datastructure 735 is not merely multiple replications of the source image atdifferent resolutions, which consists of redundant information, but itcontains unique data at the different levels of the hierarchicalrepresentation. The mathematically independent nature of the wavelettransform permits minimizing the amount of data transferred over anetwork, by requiring only the transfer of “additional data” not yettransferred to the computer from the server necessary to construct agiven image. The wavelet transforms are lossless, in that no data fromthe original source image is lost in the decomposition into thepyramidal data structure 735. Accordingly, the dynamic transfer syntaxsystem has applications for use in medical imaging and medical imagingapplications.

[0057] In one embodiment, fixed point kernels are used in the wavelettransform (i.e., decomposition processing 725). The use of fixed pointkernels generates coefficients for the pyramidal data structure thatpermit an easy implementation into a standard pixel footprint. Thewavelet transform, a spatial transform, generates a dynamic range of the“low low” component that is equal to the dynamic range of the sourceimage. Because of this characteristic, the “low low” component does notcontain overshoot or undershoot components. As a result, the use offixed point kernels is preferred because no normalization process toconvert the transformed dynamic range to the pixel dynamic range isrequired.

[0058] For this embodiment, the medical informatics system 700 directlyutilizes the transform coefficients as pixels, without re-scaling thecoefficients. The range of the high-pass components (i.e., “low high”,“high low”, and “high high” components) is the range of the input sourcedata plus two bits per coefficient. This characteristic permits mappingof all components (i.e., high and low pass components) to a given pixelfootprint.

[0059] The use of the wavelet transform to generate the pyramidal datastructure provides a scalable solution for transferring differentportions of a large data file. When the source image 715 is decomposedinto the pyramidal data structure 735, sub-images and sub-resolutionimages are extracted directly from memory of the image server 720. Theimage server then transmits only the data, in the form of physicalcoefficients, required to reconstruct the exact size of the desiredimage for display at the client. Accordingly, the multi-resolutionformat is implicit in the pyramidal data structure.

[0060] A wavelet transform is a spatial transform. In general, in aspatial transform, the information is aggregated so as to preserve thepredictability of the geometry of the source image. For example, using awavelet transform with fixed point kernels, specific coefficients of thetransform data may be identified that contribute to specific geometricfeatures of the source image (i.e., a pre-defined portion of a sourceimage is directly identifiable in the transform data). In anotherembodiment, the wavelet transforms use floating point kernels.

[0061] In other embodiments, the wavelet transform may be used togenerate multi-spectral transform data. In general, multi-spectraltransform data aggregates multi-components of the source image into avector for the transform data. Through use of multi-spectral transformdata, the wavelet transform may aggregate multi-dimensional data (e.g.,two dimensional, three dimensional, etc.) for a source image. Forexample, multi-dimensional transform data may be used to reconstruct asource image in three dimensions. Also, the multi-spectral transformdata may comprise any type of attribute for binding to the source image,such as color variations and/or non-visual components (e.g., infraredcomponents).

[0062] In general, to generate the pyramidal data structure 735, thetransform is applied across the columns, and then this transform, or adifferent transform, is applied across the rows. The selection of thetransform for decomposition processing 725 is dependent upon theparticular characteristics of the pyramidal data structure desired. Eachlevel of the pyramidal data structure is generated by recurring on thelow-pass, “low low”, of the previous higher level. This recursioncontinues until a predetermined size is obtained. For example, in oneembodiment, the lowest level in the pyramidal data structure for asource image having an aspect ratio of one-to-one consists of a low-passcomponent of 128×128. However, any granularity of resolution may begenerated for use in a pyramidal data structure without deviating fromthe spirit or scope of the invention. Also, any quadrant may be used inthe recursion process with any desired transform.

[0063]FIG. 8a illustrates an example of a pyramidal data structure. Forthis example, the source image comprises a 4K×4K image. Thedecomposition processing 725 generates, in a first iteration, a levelone Mallat structure. Specifically, as shown in FIG. 8a, a low-passcomponent, “low low”, is generated and consists of a 2K×2K sub-image.The 2K×2K sub-image is labeled in FIG. 8a as 805. The high-passcomponents, consisting of “low high”, “high high”, and “high low”,contain physical coefficient coordinates (e.g., the upper right handcoordinate for the rectangle that constitutes the “low high” componentis (4K, 0)).

[0064]FIG. 8a also illustrates a second level decomposition. The seconditeration of decomposition processing 725 operates on the low pass(i.e., “low low”), component of the level one data. For the secondlevel, the low-pass component, “low low”, consists of a 1K×1K sub-image,as labeled in FIG. 8a. FIG. 8b illustrates level three and level fourdecompositions for the 4K×4K source image of FIG. 8a. To generate thelevel three decomposition, decomposition processing 725 operates on thelevel two “low low” component (i.e., the 1K×1K image). For the levelthree transform, the low-pass component, “low low”, is a 512×512sub-image as labeled on FIG. 8a. FIG. 8b also illustrates a fourth levelof decomposition for the 4K×4K source image. For the level fourtransform, the low-pass component comprises a sub-image of 256×256pixels.

[0065] In one embodiment, the wavelet kernel comprises the waveletkernel derived from D. LeGall and A. Tabatabai, See “Sub-band coding ofdigital images using symmetric short kernel filters and arithmeticcoding techniques,” IEEE International Conference on Acoustics, Speechand Signal Processing, New York, N.Y., pp. 761-765, 1988. Any sub-bandkernel or pyramid transform could be used within the infrastructuredescribed by the dynamic transfer syntax; however, an integer kernelwith no coefficient growth in the low pass term has particularadvantages in that the low pass coefficients can be used withoutprocessing as pixels, and the transform can be inverted exactly in theinteger domain. Although floating point kernels can have superior signaltransfer characteristics, the additional processing required to usethese coefficients as pixels, and the need for additional storage toguarantee perfect reconstruction works to their disadvantage.

[0066] The kernel consists of a low pass and a high pass biorthogonalfilter. With input defined as {d_(j)} and [x] defined as the floorfunction, the forward transform is:

Low[j]=[(d _(2j) +d _(2j+1))/2]

High[2]=d _(2j) −d _(2j+1)+Poly[j]

Poly[j]=[(3*Low[j−2]−22*Low[j−1]+22*Low[j+1]−3*Low[j+2]+32)/64]

[0067] The inverse transform, used to reconstruct the image, is:

d _(2j)=Low[j]+[(High[j]−Poly[j]+1)/2]

d _(2j+1)=Low[j]−[(High[j]−Poly[j])/2]

[0068] As discussed above, the wavelet transform is a spatial transformsuch that the information is aggregated to preserve the predictabilityof the geometry of the source image. Thus, coefficient coordinatessufficient to reconstruct a desired image or sub-image at a particularlevel are readily identifiable.

[0069] A more complete description of the dynamic transfer syntax iscontained in U.S. Provisional Patent Application, entitled “FlexibleRepresentation and Interactive Image Data Delivery Protocol”, Serial No.60/091,697, inventors Paul Joseph Chang and Carlos Bentancourt, filedJul. 3, 1998, and U.S. patent application, entitled “Methods andApparatus for Dynamic Transfer of Image Data”, Ser. No. 09/339,077,inventors Paul Joseph Chang and Carlos Bentancourt, filed Jun. 23, 1999,both of which are expressly incorporated herein by reference.

[0070] Computer System:

[0071]FIG. 9 illustrates a high-level block diagram of a general purposecomputer system for implementing the user interface for the medicalinformatics system. A computer system 1000 contains a processor unit1005, main memory 1010, and an interconnect bus 1025. The processor unit1005 may contain a single microprocessor, or may contain a plurality ofmicroprocessors for configuring the computer system 1000 as amulti-processor system. The main memory 1010 stores, in part,instructions and data for execution by the processor unit 1005. If theuser interface for the medical informatics system of the presentinvention is partially implemented in software, the main memory 1010stores the executable code when in operation. The main memory 1010 mayinclude banks of dynamic random access memory (DRAM) as well as highspeed cache memory.

[0072] The computer system 1000 further includes a mass storage device1020, peripheral device(s) 1030, portable storage medium drive(s) 1040,input control device(s) 1070, a graphics subsystem 1050, and an outputdisplay 1060. For purposes of simplicity, all components in the computersystem 1000 are shown in FIG. 9 as being connected via the bus 1025.However, the computer system 1000 may be connected through one or moredata transport means. For example, the processor unit 1005 and the mainmemory 1010 may be connected via a local microprocessor bus, and themass storage device 1020, peripheral device(s) 1030, portable storagemedium drive(s) 1040, graphics subsystem 1050 may be connected via oneor more input/output (I/O) busses. The mass storage device 1020, whichmay be implemented with a magnetic disk drive or an optical disk drive,is a non-volatile storage device for storing data and instructions foruse by the processor unit 1005. In the software embodiment, the massstorage device 1020 stores the user interface software for loading tothe main memory 1010.

[0073] The portable storage medium drive 1040 operates in conjunctionwith a portable non-volatile storage medium, such as a floppy disk or acompact disc read only memory (CD-ROM), to input and output data andcode to and from the computer system 1000. In one embodiment, the userinterface for the medical informatics system software is stored on sucha portable medium, and is input to the computer system 1000 via theportable storage medium drive 1040. The peripheral device(s) 1030 mayinclude any type of computer support device, such as an input/output(I/O) interface, to add additional functionality to the computer system1000. For example, the peripheral device(s) 1030 may include a networkinterface card for interfacing the computer system 1000 to a network.

[0074] The input control device(s) 1070 provide a portion of the userinterface for a user of the computer system 1000. The input controldevice(s) 1070 may include an alphanumeric keypad for inputtingalphanumeric and other key information, a cursor control device, such asa mouse, a trackball, stylus, or cursor direction keys. In order todisplay textual and graphical information, the computer system 1000contains the graphics subsystem 1050 and the output display 1060. Theoutput display 1060 may include a cathode ray tube (CRT) display orliquid crystal display (LCD). The graphics subsystem 1050 receivestextual and graphical information, and processes the information foroutput to the output display 1060. The components contained in thecomputer system 1000 are those typically found in general purposecomputer systems, and in fact, these components are intended torepresent a broad category of such computer components that are wellknown in the art.

[0075] The user interface for the medical informatics system may beimplemented in either hardware or software. For the softwareimplementation, the user interface is software that includes a pluralityof computer executable instructions for implementation on a generalpurpose computer system. Prior to loading into a general-purposecomputer system, the user interface software may reside as encodedinformation on a computer readable medium, such as a magnetic floppydisk, magnetic tape, and compact disc read only memory (CD-ROM). In onehardware implementation, the user interface system may comprise adedicated processor including processor instructions for performing thefunctions described herein. Circuits may also be developed to performthe functions described herein.

[0076] Although the present invention has been described in terms ofspecific exemplary embodiments, it will be appreciated that variousmodifications and alterations might be made by those skilled in the artwithout departing from the spirit and scope of the invention.

What is claimed is:
 1. A method for viewing medical images in a computersystem, said method comprising the steps of: receiving in a computer atleast one study, wherein said study comprises an identification to aplurality of medical images for a patient; displaying, on an outputdisplay of said computer, a first arrangement of said medical images forsaid study including displaying a first view of said medical images,wherein a view comprises display of at least a portion of a medicalimages at a specified resolution; providing through said computer ameans to organize said first arrangement of said medical images for saidstudy to generate a second arrangement of said medical images; providingthrough said computer a means to select a second view of said medicalimages, wherein said second view comprises a view different than saidfirst view; and displaying on an output display of said computer saidsecond view of said medical image in said second arrangement.
 2. Themethod as set forth in claim 1, further comprising the step of providingthrough said computer a means to select said at least one study of apatient.
 3. The method as set forth in claim 2, wherein the step ofselecting at least one study of a patient comprises the step ofdisplaying a patient browser view, said patient browser view providing ameans to select at least one patient and at least one study for saidpatient.
 4. The method as set forth in claim 1, wherein: the step ofreceiving at least one study comprises the step of receiving a pluralityof studies for a single patient; and the step of providing to a user ameans to organize said first arrangement into a second arrangementcomprises the step of providing to a user a means to re-arrange an orderof display for said studies.
 5. The method as set forth in claim 1,wherein: the step of receiving at least one study comprises the step ofreceiving a plurality of studies for a single patient; and the step ofproviding to a user a means to organize said first arrangement into asecond arrangement comprises the step of providing to a user a means toscroll among said medical images of said studies.
 6. The method as setforth in claim 1, wherein the step of providing to a user a means toorganize said first arrangement into a second arrangement comprises thestep of providing to a user a means to re-arrange an order of displayfor said medical images within said study.
 7. The method as set forth inclaim 1, wherein the step of providing a means to select a second viewof said medical images comprises the step of providing to a user a meansto resize said medical images.
 8. The method as set forth in claim 7,wherein the step of providing to a user a means to resize said medicalimages further comprises the step of maintaining an aspect ratio betweenheight and width of said medical image.
 9. The method as set forth inclaim 1, wherein the step of providing a means to select a second viewof said medical images comprises the step of providing to a user a meansto pan a medical image such that said second view comprises a display ofa different portion of said medical images than said first view.
 10. Themethod as set forth in claim 1, wherein the step of providing a means toselect a second view of said medical images comprises the step ofproviding to a user a means to zoom in on a portion of a medical imagesuch that said second view comprises a resolution different from saidfirst view.
 11. The method as set forth in claim 1, wherein the step ofproviding to a user a means to organize said first arrangement into asecond arrangement comprises the step of providing to a user a means toscroll said medical images within a study.
 12. The method as set forthin claim 1, further comprising the steps of receiving at least twoseries; and linking said two series so as to provide simultaneous cine.13. The method as set forth in claim 1, further comprising the step ofdisplaying, in response to user selection, a report associated with saidstudy.
 14. A method for viewing and navigating a medical image on acomputer, said method comprising the steps of: storing a dynamic medicalimage so as to define data to reconstruct a plurality of portions ofsaid medical image at a plurality of resolutions, and to reconstructsaid portions of said medical image at a plurality of resolutions;displaying at least one medical image on an output display at a firstview, wherein a view comprises display of at least a portion of saidmedical image at a specified resolution; receiving user input thatdesignates a second view for said medical image, wherein said secondview comprises a view different from said first view; reconstructing, atsaid computer, said second view for said medical image; and displayingsaid second view on said output display.
 15. A computer readable mediumcomprising a plurality of instructions, which when executed by acomputer, causes the computer to perform the steps of: receiving in acomputer at least one study, wherein said study comprises anidentification to a plurality of medical images for a patient;displaying, on an output display of said computer, a first arrangementof said medical images for said study including displaying a first viewof said medical images, wherein a view comprises display of at least aportion of a medical images at a specified resolution; providing throughsaid computer a means to organize said first arrangement of said medicalimages for said study to generate a second arrangement of said medicalimages; providing through said computer a means to select a second viewof said medical images, wherein said second view comprises a viewdifferent than said first view; and displaying on an output display ofsaid computer said second view of said medical image in said secondarrangement.
 16. The computer readable medium as set forth in claim 15,further comprising the step of providing through said computer a meansto select said at least one study of a patient.
 17. The computerreadable medium as set forth in claim 16, wherein the step of selectingat least one study of a patient comprises the step of displaying apatient browser view, said patient browser view providing a means toselect at least one patient and at least one study for said patient. 18.The computer readable medium as set forth in claim 15, wherein: the stepof receiving at least one study comprises the step of receiving aplurality of studies for a single patient; and the step of providing toa user a means to organize said first arrangement into a secondarrangement comprises the step of providing to a user a means tore-arrange an order of display for said studies.
 19. The computerreadable medium as set forth in claim 15, wherein: the step of receivingat least one study comprises the step of receiving a plurality ofstudies for a single patient; and the step of providing to a user ameans to organize said first arrangement into a second arrangementcomprises the step of providing to a user a means to scroll among saidmedical images of said studies.
 20. The computer readable medium as setforth in claim 15, wherein the step of providing to a user a means toorganize said first arrangement into a second arrangement comprises thestep of providing to a user a means to rearrange an order of display forsaid medical images within said study.
 21. The computer readable mediumas set forth in claim 15, wherein the step of providing a means toselect a second view of said medical images comprises the step ofproviding to a user a means to resize said medical images.
 22. Thecomputer readable medium as set forth in claim 21, wherein the step ofproviding to a user a means to resize said medical images furthercomprises the step of maintaining an aspect ratio between height andwidth of said medical image.
 23. The computer readable medium as setforth in claim 15, wherein the step of providing a means to select asecond view of said medical images comprises the step of providing to auser a means to pan a medical image such that said second view comprisesa display of a different portion of said medical images than said firstview.
 24. The computer readable medium as set forth in claim 15, whereinthe step of providing a means to select a second view of said medicalimages comprises the step of providing to a user a means to zoom in on aportion of a medical image such that said second view comprises aresolution different from said first view.
 25. The computer readablemedium as set forth in claim 15, wherein the step of providing to a usera means to organize said first arrangement into a second arrangementcomprises the step of providing to a user a means to scroll said medicalimages within a study.
 26. The computer readable medium as set forth inclaim 15, further comprising the steps of: receiving at least twoseries; and linking said two series so as to provide simultaneous cine.27. The computer readable medium as set forth in claim 15, furthercomprising the step of displaying, in response to user selection, areport associated with said study.
 28. A computer system comprising:input device for receiving at least one study, wherein said studycomprises an identification to a plurality of medical images for apatient; output display for displaying a first arrangement of saidmedical images for said study including displaying a first view of saidmedical images, wherein a view comprises display of at least a portionof a medical images at a specified resolution; input control device forreceiving user input; processing unit, coupled to said input controldevice, for providing a means to organize said first arrangement of saidmedical images for said study to generate a second arrangement of saidmedical images, for providing a means to select a second view of saidmedical images, wherein said second view comprises a view different thansaid first view; and wherein said output display for displaying saidsecond view of said medical image in said second arrangement.